1. Field of the Invention
The present invention relates to an image region determining apparatus with an image region determining function.
2. Description of the Related Art
A digital image is frequently process through the spread of the image processing apparatus in these days. As a method of compressing such digital image data, a DCT coding method such as the JPEG method is widely used generally.
In accordance with the above-mentioned DCT coding method, an image is resolved based on the frequency component strength and a high frequency component strength is quantized roughly, and data compression is carried out. Therefore, the DCT coding system of the JPEG system is an optimal compressing method to a natural image, in which the picture element values change continuously.
However, in an actual case, an image contains characters and illustrations or drawings in addition to the natural image manipulated as a photograph object (reading object). The characters and drawings are superposed on the natural image, which is seen on the cover of a magazine. When the DCT coding method is carried out to such a natural image to compress and code the image, there is a problem that moskeyte noize is generated around the contour portion of regions of the characters and the drawing. As a result, the degradation of image quality is caused to a large extent. The reason is in that the data of the high frequency component strength of the image has been reduced so that the steep edge can not be reproduced in the contour portion of the regions of the characters and the drawings.
As the technique for solving such a problem, a method is known in which the quantization step of the DCT coefficient is controlled. In this technique, the block which contains the contour portion of the character region or drawing region can be coded in a high precision, so that a coding warp in the contour portion is improved.
However, in the method in which the above quantizing step is controlled, when the character region and the drawing region occupy a large portion of the image, there is a problem that a code quantity increases. This is because the most of the blocks are coded in the high precision to attain an enough image quality.
On the other hand, it is known that an image is divided into two regions of the character region and the natural image region, and each region is compressed by the different coding method. In accordance with this method, an entropy coding method such as a run length method is used for the character region, and a conversion coding method such that DOT and a wavelet are used for the natural image region is used.
In the above method, because each divided region (the character region and the natural image region) can be compressed in the method of coding suitable for an image characteristic for every region, the coding quality can be improved, compared with the case that the JPEG method is independently used. Also, the increase of the code quantity can be held even when the rate of the character region is increased. However, there is a problem of limitations in the processing quantity of the region separation, the size and the shape of the character which can be separated.
As the conventional example for solving the above-mentioned problem, a conventional image processing apparatus with a region determining function is shown in FIG. 1. In the conventional determining process of the character region and the photograph region in the color manuscript, the image data of the color manuscript is converted into RGB color space data represented by 3 primary colors of R (red), G (green) and B (the blue). A determination of the character region and the photograph region in the input image is carried out using the converted RGB data.
The manuscript is generally read by a CCD device or a scanner in the form of RGB data, and then each of the 3 primary color data (R, G, B) is A/D-converted. A character region and a photograph region in the input image are determined using the 8-bit data. Such a technique is known. Hereinafter, the technique will be described with reference to FIG. 1.
Referring to FIG. 1, the conventional image region determining apparatus is mainly composed of a color reducing and quantizing section 101, a constant color region clustering section 102, a color deviation variance detecting section 103, an edge-in-window detecting section 104, a region determining section 105, an image processing section 106 and a data compressing section 107.
In the above conventional image region determining apparatus, each of the RGB data of the input image is expressed in 8 bits. Such data are inputted to the color reducing and quantizing section 101, and a color reducing process is carried out therein to reduce 1 to 3 bits of the data. As this color reducing process, there is a method of simply cutting off lower 5 to 7 bits of the RGB data to be inputted and there is a method of rounding. For example, when the lower 5 bits of each 8-bit data of the RGB data are simply cut off, each of the RGB data becomes 3-bit data. At this time, the number of reduced colors is 512 (=8xc3x978xc3x978). The region clustering process is carried out to these 512 colors, and the region determination is sufficient to be carried out for every clustered region.
In the constant color region clustering section 102, an optional concerned picture element is compared with eight picture elements which are neighbor to the concerned picture element, as shown in FIG. 2 using the color reduced data obtained through the cutting off or rounding of the lower bits by the color reducing and quantizing section 101. When it is determined to be the same quantization value based on the comparing result, an integrating process is carried out as a constant color region. For example, as shown in FIG. 2, the concerned picture element is a concerned picture element P. It is supposed that eight picture elements as the peripheral picture elements which surround the concerned picture element P are a picture element P1 to a picture element P8. The picture element value of the concerned picture element P is compared with the picture element value of each of picture elements P1 to P8 in order to integrate the picture elements with respect to the same quantization value. Thus, the clustering process is carried out as the same color region.
In this case, in the constant color region clustering section 102, a region after the clustering process is composed of one constant color region or several constant color regions in the character regions and the drawing regions in which color variation is regarded to be less in the color manuscript. The characteristic quantity is found for every region to which the clustering process is carried out so that a region determination is carried out. In the conventional example, the color deviation is detected for every region in the region to which the clustering process is carried out in the color deviation variance detecting section 103. The edge portion or boundary portion of the character region in the window is detected by the edge-in-window detecting section 104. The determination of the character region or the photograph region is carried out in the region determining section 105 using the detection data.
In the detecting process in the color deviation variance detecting section 103, it is supposed that the region composed of a plurality of picture elements and integrated to have the same quantization value by the constant color region clustering section 102 is a constant color region A. In this case, the color deviation in the constant color region A is calculated using the following equation (1) shown below:
V(A)=(1/N(A))xc3x97xcexa3(Dif(C(P),C(Aav.))):n(Pxcex5A)xe2x80x83xe2x80x83(1)
where in the above-mentioned equation (1), V(A) is the variance in the color region A (deviation degree of color), N(A) is the number of picture elements in the color region A, C(P) is each 8-bit data value of RGB data in the optional concerned picture element P, C(Aav.) is an average 8-bit data value of the RGB data in the color region A.
The difference between the average C(Aav.) of the RGB data in the constant color region A and the 8-bit data value C(P) of each of the RGB colors in the concerned picture element P is calculated based on the following equation (2).
xe2x80x83Dif(C(P),C(Aav.))={(R(P)xe2x88x92R(Aav.))2+(G(P)xe2x88x92G(Aav.))2+(B(P)xe2x88x92B(Aav.))2}xc2xdxe2x80x83xe2x80x83(2)
In the above-mentioned equation (2), R(P), G(P), and B(P) represent the 8-bit data value of each of the R data, the G data, and the B data in the concerned picture element P. Also, R(Aav.), G(Aav.), and B(Aav.) are averages of the RGB data in the constant color region A. In the color deviation variance detecting section 103, the color deviation V(A) in each constant color region A is calculated based on the above-mentioned equation (2).
The detecting process in the edge-in-window detecting section 104 is carried out based on the following equation (3) to detect a steep edge around the contour portion of the input image. It should be noted that W shows the window shown in FIG. 2.
H(A)=max(H(P)):(Pxcex5A)
H(P)=Dif(C(P),C(Wav))xe2x80x83xe2x80x83(3)
In the above-mentioned equation (3), H(A): The maximum of the high frequency component strength in the color region A, and H(P): the high frequency component strength at concerned picture element P.
The high frequency component strength H(P) at the concerned picture element P is represented by use of the average of the RGB data in window of (3xc3x973) with 9 picture elements shown in FIG. 2 and the average square distance at the concerned picture element P.
The region determining section 105 carries out the determination of the character and drawing region or the photograph region in the clustered constant color region A, based on V(A) outputted from the color deviation variance detecting section 103: the variance in the color region A, and H(A) outputted from the edge-in-window detecting section 104: the maximum of the high frequency component strength in the color region A.
Here, the monochromaticity as the characteristic of the character and drawing region and the existence of the steep edge in the contour portion are determined based on two above-mentioned parameters, i.e., V(A): the variance (the color deviation) in the color region A, and H(A) outputted from the edge-in-window detecting section 104: the maximum of the high frequency component strength in the color region A. Then, a region determination is carried out based on the following condition.
The character and drawing region:
xe2x80x83V(A) less than VT1 and H(A) greater than HT1
The photograph region:
except the above condition
In the above description, VT1 is a threshold value showing the monochromaticity in the region, HT1 is a threshold value showing the existence of the high frequency component strength in the region and the region boundary. In accordance with the above condition, the region in which the monochromaticity in the region is strong and which has a steep edge is determined to be a character and drawing region.
However, the above conventional image region determining apparatus has the problems as shown below.
As the first problem, in the image that an electronic character manuscript is pasted to a natural color image in the photograph portion, it is possible to successfully carry out the region dividing process. However, when a manuscript which contains characters is read by a reading device such as a CCD and a scanner, it is not possible for the boundary portion between the character region and the background region to be clearly distinguished, unlike the image in which the characters is electronically pasted. Therefore, the middle color portion is remained. For this reason, the steep edge around the contour portion as the characteristic of the character and drawing region gets not to exist. Thus, there is a problem that the region determination can not be successfully carried out.
Also, as the second problem, when a clustering process is carried out to the constant color region, the process that the lower bits of 8 bit data are cut off is adopted as a simple quantizing process. Therefore, the constant color region is separated into the constant color regions because of difference in the highest 1 bit. Thus, there is a problem that the constant color region has been wastefully separated so that a lot of regions are produced.
Moreover, as the third problem, the above-mentioned region dividing process is carried out to the RGB data which is read by the reading device. However, the square distance as a distance space measurement of the 3-dimensional data in the RGB color space is not always coincident with the visual distance of the color by the human being on the human engineering. Therefore, when the image processing process and data compressing process are carried out using the region determination data, there is a problem that the coding efficiency in data compression is not optimized and the image quality is degraded because of the block warp caused In the band compression.
In conjunction with the above description, an image processing apparatus is disclosed in Japanese Laid Open Patent Application (JP-A-Heisei 7-99581). In this reference, the image processing apparatus is composed of an image input section (1) and a color image/monochromatic image converting section (2) for separating an image region. A binary value producing section (3) converts of picture element values of the converted monochromatic image into binary values. A reducing section (4) reduces the binary image. A boundary extracting section (5) extracts a boundary of the region of components such as the binary image and a continuous tone image of the input image. An image kind determining section (6) determines a picture kind of a partial regional defined by the extracted boundary. The image processing apparatus is further composed of a data compressing section (7). A preprocessing is carried out for an effective pattern determination, aiming for an image edge and the generation frequency of a black picture element pattern. The image kind determination is carried out to the preprocessed data by a neural network.
Also, an image processing apparatus is disclosed in Japanese Laid Open Patent Application (JP-A-Heisei 7-203198). In this reference, the thickness of a character/line in an image is determined from an RGB signal by a thickness determining section (114) of a black character determining section (113). A contour data of the character/line is detected by an edge detecting section (115), and a chroma data is detected by a chroma detecting section (116). A thickness determination signal is corrected such that the thickness of the character and line is continuously changed, when image processing should be carried out based on the combination of the contour data and the chroma data.
Also, an image region separating apparatus is disclosed in Japanese laid Open Patent Application (JP-A-Heisei 8-237475). In this reference, various processes are carried out to a window of Mxc3x97N picture elements including a concerned picture element. The processes are such as a net point feature quantity extracting process (S1), a maximum signal level difference detecting process (S3), an edge detecting process (S6) through pattern matching, a change counting process (S7) and a percentage count (S8). The determination of net points, character and photograph is carried out based on the processing results.
Also, an image reading apparatus is disclosed in Japanese Laid Open Patent Application (JP-A-Heisei 11-88705). In this reference, the image reading apparatus is composed of window setting means (14), edge quantity calculating means (18), filter selecting means (19), filter calculating and selecting means (20) of calculating an output candidate value, and output value selecting means (24) for selecting an output candidate value. Because an optimal high-pass filter is operated based on the edge quantity, an image of high quality can be outputted. In the image reading apparatus, the degradation of the image quality read out by the image reading apparatus such as a scanner can be prevented.
Also, an image processing apparatus is disclosed in Japan Patent No. 2,672,553. In this reference, a tone process is carried out to an image data with respect to a plurality of color components of an image, and an image data is outputted with respect to the plurality of color components in the image processing apparatus. First determining means determines in units of color components whether or not a region of the image is a net point region. Second determining means determines as a net point region the region of the plurality of other color components corresponding to a region of one color component determined by the first determining means. Tone process means carries out a tone process for the net point region to the region determined as the net point region by the second determining means. Also, third determining means determines whether a region is a character region or a photograph region for every color component. Second tone process means carries out a tone process for the character region to output image data. Third tone process means carries out a tone process for the photograph region to output image data. Selecting means selects one of the second tone process means and the third tone process means based on the determining result of the third determining means to the regions other than the net point region determined by the second determining means.
Therefore, an object of the present invention is to provide an image processing apparatus with an image region determining function, in which a middle color in the boundary portion between a character region and a photograph region can be corrected.
Another object of the present invention is to provide an image processing apparatus with an image region determining function, in which the process load in a region determination stage can be mainly reduced.
Still another object of the present invention is to provide an image processing apparatus with an image region determining function, in which the number of division regions can be diminished.
Yet still another object of the present invention is to provide an image processing apparatus with an image region determining function, in which image quality can be improved by an image process and image degradation due to block warp in a data compressing process can be reduced.
In order to achieve an aspect of the present invention, an image processing apparatus includes an edge processing section, a region data producing section, and a region determining section. The edge processing section enhances an edge portion of a first region of an image in units of picture elements to produce an enhanced image. The image includes the first region and a second region which are mixed, and picture elements of the image are expressed to as RGB data. The region data producing section divides the enhanced image into regions to output region data indicative of each of the regions, variance data of each of the regions and contour edge data indicative of a contour of each of the regions. The region determining section determines the first region in the enhanced image based on the region data, the variance data and the contour edge data.
The edge processing section selectively sets a picture element value of each of picture elements of the image as a concerned picture element to that of one of neighbor picture elements of the picture element in units of picture elements of the image, to produce the enhanced image. In this case, the edge processing section may carry out an enhancing process to set the picture element value of the concerned picture element to the picture element value of a first one of the neighbor picture elements when the concerned picture element has a middle color between the first neighbor picture element and a second one of the neighbor picture elements in a direction opposite to a direction of the first neighbor picture element with respect to the concerned picture element. In this case, the edge processing section carries out the enhancing process to set the picture element value of the concerned picture element to the picture element value of the first neighbor picture element, when the concerned picture element has the middle color, and when a first color difference between the concerned picture element and the first neighbor picture element is smaller than a second color difference between the concerned picture element and the second neighbor picture element. Also, the edge processing section carries out the enhancing process to set the picture element value of the concerned picture element to the picture element value of the second neighbor picture element, when the concerned picture element has the middle color, and when the first color difference is equal to or larger than the second color difference between the concerned picture element and the second neighbor picture element. In this case, the color difference between the first and second neighbor picture elements is the largest among the color differences in the other opposing directions. The edge processing section may carry out the enhancing process, when a color difference between the first and second neighbor picture elements is equal to or larger than a first predetermined value. Also, the edge processing section may carry out the enhancing process, when a ratio of a third color difference between picture elements outside of the first and second neighbor picture elements in the opposing directions to a third color difference between the first and second neighbor picture elements is equal to or larger than a second predetermined value.
Also, the region data producing section may include a converting section, a density dividing section and a hue dividing section. The converting section converts the RGB data of each of the picture element of the enhanced image into density data and hue data. The density dividing section divides the enhanced image into density regions with respect to the density data to output density region data indicative of each of the density regions, density variance data of each of the density regions and density contour edge data indicative of a contour of each of the density regions. The hue dividing section which divides the enhanced image into hue regions with respect to the hue data to output hue region data indicative of each of the hue regions, hue variance data of each of the hue regions and hue contour edge data indicative of a contour of each of the hue regions.
In this case, the density dividing section desirably includes a density reducing and quantizing section, a constant density clustering section, a density variance detecting section, a density edge-in-window detecting section and a density edge continuity confirming section. The density reducing and quantizing section quantizes the density data. The constant density clustering section divides the enhanced image into density regions with respect to the density data to output the density region data for each of the density regions. The density variance detecting section which calculates the density variance data of each of the density regions to detect density uniformity of each density region. The density edge-in-window detecting section which detect steepness of an edge in a window. The density edge continuity confirming section which confirms continuity of the edge from the density data and the detected steepness to produce the density contour edge data to output the density contour edge data. In this case, the density reducing and quantizing section desirably rounds lower bits of the density data for the quantization.
Also, the hue dividing section may include a hue reducing and quantizing section, a constant hue clustering section, a hue variance detecting section, a hue edge-in-window detecting section, and a hue edge continuity confirming section. The hue reducing and quantizing section quantizes the hue data. The constant hue clustering section divides the enhanced image into hue regions with respect to the hue data to output the hue region data for each of the hue regions. The hue variance detecting section which calculates the hue variance data of each of the hue regions to detect hue uniformity of each hue region. The hue edge-in-window detecting section which detect steepness of an edge in a window. The hue edge continuity confirming section which confirms continuity of the edge from the hue data and the detected steepness to produce the hue contour edge data to output the hue contour edge data. In this case, the hue reducing and quantizing section desirably rounds lower bits of the hue data for the quantization.
Also, the region determining section may include a density region determining section, a hue region determining section and an integrating section. The density region determining section determines objective hue regions among the density regions based on density variance data of the variance data and density contour edge data of the contour edge data. The hue region determining section which determines objective density regions based on hue variance data of the variance data and hue contour edge data of the contour edge data. The integrating section integrates the determined objective density regions by the density region determining section and the determined objective hue regions by the hue region determining section to produce the first region. In this case, when the integrating section adds first region indication data and second region indication data to the first region, the region determining section may further include an image process region processing section which outputs picture element region indication data from the first region indication data and second region indication data in units of picture elements. The image processing apparatus further includes an image processing section which processes the image based on the picture element region indication data from the image process region processing section. Further, the region determining section may further include a coding process region processing section which processes the first region indication data and second region indication data in units of block to output block region indication data. At this time, the image processing apparatus may further include a data compressing section which compresses the processed image from the image processing section based on the block region indication data.
In another aspect of the present invention, an image processing method is attained by enhancing an edge portion of a first region of an image in units of picture elements to produce an enhanced image, wherein the image includes the first region and a second region which are mixed, and picture elements of the image are expressed to as RGB data; by dividing the enhanced image into regions to output region data indicative of each of the regions, variance data of each of the regions and contour edge data indicative of a contour of each of the regions; and by determining the first region in the enhanced image based on the region data, the variance data and the contour edge data.
The enhancing may be attained by selectively setting a picture element value of each of picture elements of the image as a concerned picture element to that of one of neighbor picture elements of the picture element in units of picture elements of the image, to produce the enhanced image. More specifically, the enhancing may be attained by carrying out an enhancing process to set the picture element value of the concerned picture element to the picture element value of a first one of the neighbor picture elements when the concerned picture element has a middle color between the first neighbor picture element and a second one of the neighbor picture elements in a direction opposite to a direction of the first neighbor picture element with respect to the concerned picture element.
In this case, the enhancing is attained by carrying out the enhancing process to set the picture element value of the concerned picture element to the picture element value of the first neighbor picture element, when the concerned picture element has the middle color, and when a first color difference between the concerned picture element and the first neighbor picture element is smaller than a second color difference between the concerned picture element and the second neighbor picture element, and to set the picture element value of the concerned picture element to the picture element value of the second neighbor picture element, when the concerned picture element has the middle color, and when the first color difference is equal to or larger than the second color difference between the concerned picture element and the second neighbor picture element. In this case, the color difference between the first and second neighbor picture elements is the largest among the color differences in the other opposing directions.
Also, the enhancing may be attained by carrying out the enhancing process, when a color difference between the first and second neighbor picture elements is equal to or larger than a first predetermined value. Also, the enhancing may be attained to carry out the enhancing process, when a ratio of a third color difference between picture elements outside of the first and second neighbor picture elements in the opposing directions to a third color difference between the first and second neighbor picture elements is equal to or larger than a second predetermined value.
Also, the dividing may be attained by converting the RGB data of each of the picture element of the enhanced image into density data and hue data; by carrying out a first clustering operation of the enhanced image into density regions with respect to the density data to output density region data indicative of each of the density regions, density variance data of each of the density regions and density contour edge data indicative of a contour of each of the density regions; and by carrying out a second clustering operation of the enhanced image into hue regions with respect to the hue data to output hue region data indicative of each of the hue regions, hue variance data of each of the hue regions and hue contour edge data indicative of a contour of each of the hue regions.
In this case, the carrying out a first clustering operation may be attained by quantizing the density data; by clustering the enhanced image into density regions with respect to the density data to output the density region data for each of the density regions; by calculating the density variance data of each of the density regions to detect density uniformity of each density region; by detecting steepness of an edge in a window; and by confirming continuity of the edge from the density data and the detected steepness to produce the density contour edge data to output the density contour edge data. In this case, the quantizing desirably includes rounding lower bits of the density data for the quantization.
Also, the carrying out a first clustering operation may be attained by quantizing the hue data; by dividing the enhanced image into hue regions with respect to the hue data to output the hue region data for each of the hue regions; by calculating the hue variance data of each of the hue regions to detect hue uniformity of each hue region; by detecting steepness of an edge in a window; and by confirming continuity of the edge from the hue data and the detected steepness to produce the hue contour edge data to output the hue contour edge data. In this case, the quantizing desirably includes: rounding lower bits of the hue data for the quantization.
Also, the determining the first region may be attained by determining objective hue regions among the density regions based on density variance data of the variance data and density contour edge data of the contour edge data; by determining objective density regions based on hue variance data of the variance data and hue contour edge data of the contour edge data; and integrating the determined objective density regions by the density region determining section and the determined objective hue regions by the hue region determining section to produce the first region.
In this case, the integrating may include: adding first region indication data and second region indication data to the first region. In this case, when the determining the first region further include: outputting picture element region indication data from the first region indication data and second region indication data in units of picture elements, the image processing method may further include: processing the image based on the picture element region indication data from the image process region processing section.
Further, when the determining the first region further include: processing the first region indication data and second region indication data in units of block to output block region indication data, the image processing method further include: compressing the processed image from the image processing section based on the block region indication data.
In order to achieve still another aspect of the present invention, a recording medium in which a program is stored for any of the above image processing methods.